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LONG-TERM  GOAL 

The  long-term  goals  of  this  project  are:  (i)  explore  and  test  new  techniques  that  can  improve  the 
application  of  the  Coupled  Ocean/Atmosphere  Mesoscale  Prediction  System  (COAMPS™)1  high- 
resolution  nested  grids,  (ii)  to  develop  new  mesoscale  model  verification  techniques  to  objectively 
validate  the  model  forecast,  (iii)  to  perform  model  validation,  and  (iv)  use  the  validation  results  as 
guidance  to  improve  COAMPS  high-resolution  grid  forecasts. 

OBJECTIVES 

Compare  validation  results  on  COAMPS  coarse  and  fine  grids.  Investigate  methods  and  type  of 
observations  that  can  be  used  for  validation  on  high-resolution  mesoscale  model  forecasts.  Improve 
flexibility  of  COAMPS  fine  grid  initialization  and  displacements. 

APPROACH 

Validation  of  the  model  Quantitative  Precipitation  forecast  (QPF),  wind  forecast,  and  other  model 
statistics  are  important  to  gauge  the  performance  of  COAMPS.  Our  approach  is  to  examine  the 
COAMPS  cold  and  wann  season  QPF  statistics  to  establish  baseline  statistics  for  future  comparison. 
Case  studies  are  performed  to  understand  the  causes  of  model  precipitation  bias  using  the  NCEP  rain 
gauge  analysis  over  the  CONUS  region.  Rule-based  algorithms  are  developed  to  test  the  composite 
verification  method  for  partially  observed  high  wind  events  near  the  ocean  surface.  The  composite 
technique  is  a  relative  simple  method  compared  to  more  complex  pattern  shifting.  It  relaxes  the 
observation  requirements  on  any  one  event  and  allows  a  partially  observed  event  to  be  smoothly 
incorporated  into  a  coherent,  statistically  meaningful  comparison. 

To  improve  the  application  of  COAMPS  high  resolution  nested  grids,  our  approach  is  to  incorporate  a 
delayed  nest  option  that  uses  the  moving  nest  algorithm.  The  delayed  nest  can  reduce  the  overall  wall 
time  needed  to  generate  the  forecast  and  allow  even  finer  grids  to  track  features  of  interest. 
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WORK  COMPLETED 


1.  Validated  the  COAMPS  2002  cold  season  quantitative  precipitation  forecasts. -Used  the  event 

verification  technique  to  obtain  the  observed  and  the  model  precipitation  distribution 
climatology  in  addition  to  the  grid  averaged  equitable  threat  and  bias  scores. 

-Perfonned  one-week  case  studies  to  investigate  the  causes  for  the  model  precipitation 
deficiency. 

2.  Completed  the  wind  event-based  verification  study. 

-Developed  the  algorithm  for  high  wind  event  composites  near  the  ocean  surface. 

-Validated  COAMPS  wind  forecast  over  the  Mediterranean  and  eastern  Pacific  Ocean. 

3.  Developed  the  delayed  nested  grid  ability. 

RESULTS 

COAMPS  QPF  Validation:  The  COAMPS  precipitation  statistics  for  the  2001  cold  season  were 
computed  using  the  27  km  operation  CONUS  forecasts  with  eight  different  thresholds  of  rain  rate.  The 
time  period  is  from  mid  November  2001  through  30  April  2002.  Due  to  various  data  collecting 
problems,  1 1 1  days  statistics  are  actually  used  for  the  average. 


Precipitation  Threshold  (imfday) 


Figure  1.  COAMPS  bias  (left)  and  equitable  threat  scores  (right)  for  precipitation  thresholds  of 
0.25  to  75  mm/day.  The  blue  line  with  diamonds  represents  the  0-24  hour  forecast  and  the  red  line 

with  squares  represents  the  24-48  hour  forecast. 


Note  that  in  Fig.  1  (left),  the  comparison  of  the  averaged  COAMPS  rain  area  coverage  (represented  by 
the  bias  score)  with  the  observations  indicates:  for  rain  amounts  <  35  mm/day,  the  area  was  over¬ 
predicted,  i.e.,  bias>l;  for  rain  amounts  >35mm/day  the  rain  area  was  under-predicted.  Comparison  of 
the  0-24  and  24-48  hour  forecasts  bias  scores  shows  the  model  has  a  positive  precipitation  bias  with 
increasing  forecast  time.  The  COAMPS  relative  humidity  field  also  showed  the  same  bias  trend 
indicating  the  precipitation  problem  was  related  to  an  increase  in  the  modeled  moisture  with  time. 
Since  the  bias  scores  are  close  to  one  for  a  rain  rate  <50  mm/day,  the  low  threat  score  in  Fig.  1  (right) 
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seems  to  indicate  the  COAMPS  forecasts  do  not  produce  enough  “exact”  rain  coverage  suggesting  a 
possible  phase  shift  problem.  For  a  rain  rate  >75  mm/day,  both  the  bias  and  threat  scores  are  low 
suggesting  the  model  does  not  forecast  enough  heavy  precipitation.  Further  examination  shows  that 
less  than  20%  of  the  total  model  forecast  rain  is  convective  for  the  rain  rate  >  35  mm/day. 

Additional  model  runs  were  performed  for  a  one-week  period  (1200  UTC  25  January  to  1  February 
2002)  to  investigate  the  model  precipitation  bias.  Two  9km  domains  are  added  to  the  operational 
CONUS  domain.  The  precipitation  for  the  9  km  domains  is  explicitly  resolved  by  the  model 
microphysics  scheme  (e.g.  the  convective  scheme  is  not  turned  on).  A  bench  run  using  the  current 
version  of  microphysics  and  a  sensitivity  ran  using  the  improved  microphysics  (Schmidt  2001)  are 
examined.  The  one-week  case  study  shows  that  increasing  resolution  improved  the  model  QPF.  When 
using  the  new  microphysics,  even  more  QPF  improvements  are  obtained.  Similar  to  the  27  km  cold 
season  QPF  results,  the  case  studies  indicate  possible  precipitation  deficiency  from  the  COAMPS  con¬ 
vective  scheme. 
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Figure  2.  Comparison  of  COAMPS  equitable  threat  scores  between  the  standard  (bench)  and  the 
improved  microphysics  (moist)  runs  (a)  at  27  km  resolution  over  the  CONUS  region. 

Wind  Event  Composite  Verification :  A  rules-based  algorithm  was  developed  to  define  unique, 
contiguous  high  wind  events  in  each  forecast.  A  sub-domain  was  defined  on  the  27  km  model  grid  in 
which  the  algorithm  searched  for  high  wind.  Once  an  event  was  identified,  all  surrounding  data  were 
transferred  to  a  3 1X3 1  point  relative  grid  with  the  same  grid  spacing  as  the  model.  The  center  of  the 
forecast  event  as  defined  by  its  “center  of  mass”  was  positioned  at  the  center  of  the  relative  grid.  At 
that  point,  all  available  observational  data  were  also  positioned  on  the  relative  grid.  Model  data  were 
then  templated  by  the  available  observations,  such  that  all  forecasts  outside  of  the  SSM/I  swath  were 
cut  from  the  set  (Fig.  3).  Transferring  the  events  to  the  relative  grid  effectively  synchronizes  them 
about  a  common  central  point.  With  the  composite  method,  information  regarding  where  the  model 
predicts  events  with  respect  to  when  and  where  the  events  occur  can  then  be  obtained.  The  rate  of 
under-versus-over  prediction  can  be  derived  as  well  as  any  consistent  phase  errors. 

The  composite  technique  is  applied  to  validate  the  COAMPS  wind  forecast  at  the  Mediterranean  and 
eastern  Pacific.  The  results  showed  direct  correlations  between  the  model  and  observed  speed 
distributions  (Fig. 4)  are  quite  high  through  the  forecast  period  for  the  Mistral.  The  bias  and  RMS 
errors,  though  steadily  increasing,  stay  relatively  low  through  66  hours,  indicating  that  the  distributions 
are  relatively  well  synchronized.  In  general,  Mistral  winds  were  almost  always  observed  when  the 
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model  predicted  them,  and  were  almost  always  correctly  forecast  when  they  were  observed.  However, 
the  core  winds  in  the  model  were  shifted  to  the  southwest  by  about  50  km.  Also,  winds  at  the  eastern 
and  western  edges  of  the  Mistral  were  not  as  well  forecast,  indicating  that  the  model  does  not 
necessarily  predict  the  exact  shape  of  the  Mistral. 


Figure  3.  Schematic  depicting  the  data  collection  strategy  for  the  event  composites. 


Figure  4.  Relative  grid-total  values  of  RMS  and  bias  (FCST-SSM/I)  in  m  s'1  are  plotted  for  each 
forecast  hour  as  dashed  and  solid  lines,  and  the  correlation  coefficient  between  the  FCST  and 
SSM/I  speed  distributions  is  plotted  as  the  short-dashed  line. 

Over  the  eastern  Pacific,  the  model  tends  to  produce  too  many  strong  southerly  wind  events  associated 
with  Pacific  cyclones.  On  the  other  hand,  when  a  southerly  wind  event  is  actually  observed  the  model 
does  a  good  job  with  both  the  position  and  magnitude  of  the  event.  Northerly  wind  events  show  similar 
but  less  severe  tendencies.  There  is  a  modest  over-prediction  of  northerly  wind  events,  but  when  a 
wind  event  actually  exists  the  model  handles  it  well  through  most  of  the  forecast.  The  northerly  wind 
events  exhibited  very  little  phase  error. 
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Delayed  Nest:  A  1.34  wall  time  speed-up  for  a  36-hour  model  forecast  (nest  2  delayed  12  hours)  was 
achieved  when  computations  on  nest  2  were  not  begun  until  12  hours  into  the  forecast.  Two  36-hour 
COAMPS  simulations  were  performed  on  a  synoptic  high  pressure  system  to  examine  the  accuracy  of 
the  COAMPS  delayed  nest  algorithm.  The  delayed  nest  run  was  able  to  reproduce  similar  results 
compared  to  the  control  run. 

IMPACT/APPLICATIONS 

The  base-line  QPF  statistics  established  through  this  study  can  be  used  to  gauge  future  model 
improvements.  The  QPF  validation  results  indicated  a  possible  precipitation  deficiency  from  the 
COAMPS  convective  scheme.  The  composite  verification  technique  can  be  easily  adapted  to  validate 
other  type  of  meteorological  events. 

TRANSITIONS 

New  algorithms  for  the  delayed  nest  has  been  transitioned  to  6.4  programs  (PE  0602435N  and  PE 
0603207N)  for  applications  within  COAMPS  and  for  subsequent  transition  to  Fleet  Numerical 
Meteorology  and  Oceanography  Center  (FNMOC)  and  Regional  Naval  Meteorology  and 
Oceanography  Centers  for  operational  use. 

RELATED  PROJECTS 

A  related  6.2  project  within  PE  0602435N  is  BE-35-2-44,  Advanced  Moist  Physics  Modeling. 

SUMMARY 

We  have  conducted  the  2001  cold  season  COAMPS  QPF  validation  over  the  CONUS  area.  The  results 
were  similar  to  the  previous  study  on  the  summer  month  QPF  statistics  in  1999.  Both  results  showed 
not  enough  heavy  precipitation  is  occurring  for  grid  resolutions  greater  than  27  km.  When  the  grid 
resolution  was  increased  to  9  km  and  the  precipitation  was  explicitly  resolved  using  the  microphysical 
scheme  only,  the  model  QPF  was  improved.  The  simulation  with  the  new  explicit  scheme  had  the  best 
QPF  scores.  Using  the  composite  verification  method,  the  bulk  properties  of  the  forecast  and  the 
observed  high  wind  climatology  can  be  reliably  estimated.  The  COAMPS  Mistral  forecast  has  a  very 
good  skill  up  to  66  hour  forecast  time. 
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